When moving images or animation captured at a high shutter speed is displayed by using a display device, such as a project or a display, image degradation is often seen in which motion of a moving object in the images is discontinuously displayed and a viewer of the images perceives multiple images. This phenomenon is degradation of image quality typically called “jerkiness”.
On the other hand, when moving images captured at a low shutter speed, such as an open shutter, are displayed, a loss of details of a subject or a blurred edge is often seen due to an effect of motion blur. This phenomenon is degradation of image quality called “blur”.
Principles of occurrence of the jerkiness and blur are described below with reference to FIGS. 1 to 3. As a perceptual characteristic of the human, it is known that the human perceives light entered the eyes as a value integrated for a certain time period. FIGS. 1 to 3 illustrate simulations of how an object is viewed from a viewer based on this perceptual characteristic.
FIG. 1 illustrates an example of how a stationary object and a moving object are viewed in the real world.
In FIG. 1, (1) shows a temporal change of a stationary object 11 and a moving object 12, in which the horizontal axis indicates a position (x) and the vertical axis indicates time (t). In FIG. 1, (2) shows a simulation of a perceptual state of a viewer viewing the stationary object 11 and the moving object 12. The viewer views the moving object 12 by tracking it (tracking vision) or views the moving object 12 by fixing a viewpoint without tracking the object (fixed vision). Thus, there are perceptual states in two different viewing patterns. The two patterns are shown as: (a) tracking vision and (b) fixed vision.
As shown in (a) tracking vision in FIG. 1 (2), moving object perception information a12 indicates how the moving object 12 is viewed when the viewer views the moving object 12 by tracking it. This is the same as fixed object perception information b11 indicating how the fixed object 11 is viewed in (b) fixed vision in FIG. 1 (2). In this way, when the viewer views the moving object 12 by tracking it, how the moving object 12 is viewed for the viewer is the same as how the fixed object 11 is viewed in the fixed vision.
On the other hand, as shown in (b) fixed vision in FIG. 1 (2), moving object perception information b12 indicates how the moving object 12 is viewed when the viewer views the moving object 12 by fixing the viewpoint. In this case, the viewer percepts a change of the moving object that continuously moves, and the viewer does not feel unnaturalness.
FIG. 2 illustrates occurrence of jerkiness that is perceived by the viewer when moving images or animation captured at a high shutter speed is displayed by using a display device, such as a projector or a display. That is, the jerkiness is a phenomenon where motion of a moving object included in images is discontinuously displayed and the viewer viewing the images perceives multiple images.
FIG. 2 illustrates a simulation of a view from the viewer in a case where the moving object in the real world shown in FIG. 1 is captured by using a high shutter speed and the images thereof are displayed on a display device that is refreshed at 60 Hz. In FIG. 2, (1) shows a change in display position on the display device of a displayed stationary object 21 and a displayed moving object 22. The vertical axis indicates time (t), which is marked at refresh intervals ( 1/60 sec) of the display device. The horizontal axis indicates a display position (x).
In FIG. 2, (2) shows a simulation of a perceptual state of the viewer viewing the displayed stationary object 21 and the displayed moving object 22 displayed on the display device. The viewer views the displayed moving object 22 by tracking it (tracking vision) or views the displayed moving object 22 by fixing a viewpoint without tracking the object (fixed vision). Thus, there are perceptual states in two different viewing patterns. The two patterns are shown as: (a) tracking vision and (b) fixed vision.
As shown in FIG. 2 (2) (a), a view (a22) in a case where the viewer views the displayed moving object 22 displayed on the display device with the tracking vision is the same as the view (a12) of the tracking vision illustrated in FIG. 1 (2) (a). That is, the perception for the viewer is the same as in the case of viewing the stationary object with the fixed vision.
On the other hand, when the viewer views the displayed moving object 22 displayed on the display device with the fixed vision, the displayed moving object 22 perceived by the viewer's vision is different from that in the real world, but discrete (discontinuous) movement is viewed (b22), as shown in FIG. 2 (2) (b). As a result, the viewer perceives the moving object displayed on the display device as multiple images based on the perceptual characteristic that the human perceives light entered the eyes as a value integrated for a certain time period.
The viewer recognizes a single object as a plurality of objects disadvantageously. This phenomenon is called “degradation due to jerkiness”. In principle, the degradation due to jerkiness is more likely to occur as the movement velocity of an object is higher. Also, the degradation due to jerkiness is more likely to occur as the frame rate on the display device is lower and is less likely to occur as the frame rate is higher. Furthermore, the degradation due to jerkiness is more likely to occur typically in a part where a change in spatial brightness is significant, in other words, a part where spatial contrast is high.
FIG. 3 illustrates occurrence of blur perceived by the viewer in a case where moving images or animation captured at a low shutter speed, such as an open shutter, is displayed by using a display device, such as a projector or a display. The blur is a phenomenon in which a loss of details of a subject or a blurred edge occurs due to an effect of motion blur.
FIG. 3 illustrates a simulation of a view from the viewer in a case where the moving object in the real world shown in FIG. 1 is captured by using a low shutter speed and the images thereof are displayed on a display device refreshed at 60 Hz. In FIG. 3, (1) shows a change in display positions on the display device of a displayed stationary object 31 and a displayed moving object 32. The vertical axis indicates time (t), which is marked at refresh intervals ( 1/60 sec) of the display device. The horizontal axis indicates a display position (x).
In FIG. 3, (2) shows a simulation of a perceptual state of the viewer viewing the displayed stationary object 31 and the displayed moving object 32 displayed on the display device. The viewer views the displayed moving object 32 by tracking it (tracking vision) or views the displayed moving object 32 by fixing a viewpoint without tracking the object (fixed vision). Thus, there are perceptual states in two different viewing patterns. The two patterns are shown as: (a) tracking vision and (b) fixed vision.
As shown in FIG. 3 (2) (b), in a case where the viewer views the displayed moving object 32 displayed on the display device with the fixed vision (b32), the view is the same as that in the fixed vision (b12) shown in FIG. 1 (2) (b) described above with reference to FIG. 1. The viewer perceives that the moving object continuously moves and does not feel unnaturalness.
On the other hand, in a case where the viewer views the displayed moving object 32 with the tracking vision, the object is perceived by the viewer as a blurred image that is different from the fixed vision of the stationary object, as shown in a view (a32) in FIG. 3 (2) (a). This is because, as shown in the displayed moving object 32 in FIG. 3 (1), motion of the moving object during long exposure based on a low shutter speed is recorded in one frame during imaging, and this moving object is displayed in a band shape in one frame. This phenomenon is called “degradation due to blur”.
As described above, occurrence of degradation due to jerkiness and occurrence of degradation due to blur are contradictory to each other regarding a shutter speed during imaging. Thus, degradation due to any of them is conspicuous when a simple shutter control is performed.
The jerkiness is known as degradation of image quality that is likely to occur when moving images are displayed by converting its frame rate to a frame rate different from that of original images. For example, as means for converting a high frame rate of original moving images to a lower frame rate so as to display the images, a method of simply thinning frames in accordance with the number of frames to be obtained is typically used. In this case, however, jerkiness is likely to occur because the original moving images are captured in short exposure.
As a method for suppressing the jerkiness that occurs at conversion of a frame rate, there is known a method for simply averaging frames of original moving images in accordance with the number of frames to be obtained, instead of converting the frame rate by thinning frames. This frame averaging method enables suppression of jerkiness. However, this method corresponds to generating images of a low frame rate or a low shutter speed due to the averaging based on images of a high frame rate corresponding to a high shutter speed. As a result, degradation due to blur is conspicuous disadvantageously.
On the other hand, in a case where a low frame rate of original moving images is converted to a higher frame rate, the simplest method is displaying existing adjacent frames a plurality of times for the frames that do not exist temporally. However, such frame rate conversion causes motion of a moving subject to be discrete, and thus jerkiness is likely to occur.
As a method for suppressing occurrence of jerkiness in the above-described case where a frame rate converting process is performed to generate images of a high frame rate based on original images of a low frame rate, there is known a method for generating frames that do not temporally exist by using motion compensation in interpolation so as to generate moving images of a high frame rate (for example, Patent Document 1). By using this method, degradation due to jerkiness can be reduced.    [Patent Document 1] Japanese Unexamined Patent Application Publication No. 11-112939